Skip to content

docs: improve per flavor traceability - #1509

Open
Ron (rjaegers) wants to merge 15 commits into
mainfrom
docs/improve-per-flavor-traceability
Open

Ron (rjaegers) wants to merge 15 commits into
mainfrom
docs/improve-per-flavor-traceability

Conversation

@rjaegers

@rjaegers Ron (rjaegers) commented Oct 27, 2026 •

Copy link
Copy Markdown
Member

🚀 Hey, I have created a Pull Request

Description of changes

This pull request introduces several improvements to requirements traceability, acceptance test configuration, and documentation structure. The most significant changes include enhanced traceability reporting in documentation, improved configuration and selection of acceptance test scenarios by tag, and the relocation and deduplication of requirement definitions. There are also updates to the Playwright and BDD test infrastructure to better support these changes.

Requirements traceability and documentation:

  • The requirements traceability matrix template (requirements-traceability-matrix.md.j2) now lists each requirement with its declared verification tests and their source locations, clarifies that a trace link does not guarantee test execution or passing, and provides improved traceability reporting and summary. [1] [2] [3]
  • The software verification plan template (software-verification-plan.md.j2) now displays the source location for each test linked to a requirement.
  • The ADR template updates a requirements link to point to the new requirements directory.

Acceptance test configuration and selection:

  • Acceptance test workflows (wc-build-push-test.yml, build-push-test.yml, wc-acceptance-test.yml) now pass and consume an acceptance-test-tag input, enabling selection of test scenarios by tag (e.g., @flavor:cpp). The acceptance test job now checks for this tag and passes it as an environment variable. [1] [2] [3] [4] [5]
  • The Playwright configuration (playwright.config.ts) and related scripts are updated to select features from the new requirements directory and filter scenarios by the ACCEPTANCE_TEST_TAG environment variable. [1] [2]

Requirements and test definitions:

  • The ARM Cortex compilation requirement is moved from test/embedded-cpp/features/compilation.feature (now removed) to docs/requirements/compilation.feature, consolidating requirement definitions in the documentation. [1] [2]
  • Several Rust integration tests in integration-tests.bats are now explicitly linked to requirements with SBDL annotations, improving traceability. [1] [2] [3] [4] [5] [6]

Document generation and validation:

  • The document generation workflow now removes a sample SBDL file before building and validating the SBDL model to ensure clean builds.

✔️ Checklist

  • I have followed the contribution guidelines for this repository
  • I have added tests for new behavior, and have not broken any existing tests
  • I have added or updated relevant documentation
  • I have verified that all added components are accounted for in the SBOM
  • I understand the image size delta and agree the functionality justifies it

Copilot AI balanced review requested due to automatic review settings October 27, 2026 13:22
@github-actions

github-actions Bot commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

📦 Container Size Analysis

Note

Comparing ghcr.io/philips-software/amp-devcontainer-base:edge ➔ ghcr.io/philips-software/amp-devcontainer-base:pr-1509

📈 Size Comparison Table

OS/Platform Previous Current Change Trend
linux/amd64 95.71 MB 95.72 MB +6.69 kB (+0.01%) 🔼
linux/arm64 92.43 MB 92.43 MB 559 B (0%) 🔽

@rjaegers Ron (rjaegers) changed the title Docs/improve per flavor traceability docs: improve per flavor traceability Oct 27, 2026
@github-actions

github-actions Bot commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

✅⚠️MegaLinter analysis: Success with warnings

Descriptor Linter Files Fixed Errors Max errors Warnings Elapsed time
✅ ACTION actionlint 23 0 0 0.3s
✅ DOCKERFILE hadolint 4 0 0 0.51s
✅ JSON npm-package-json-lint yes no no 0.52s
✅ JSON prettier 53 14 0 0 3.51s
✅ JSON v8r 53 0 0 16.91s
✅ MARKDOWN markdownlint 13 0 0 0 1.24s
✅ MARKDOWN markdown-table-formatter 13 0 0 0 0.31s
✅ REPOSITORY betterleaks yes no no 1.32s
✅ REPOSITORY checkov yes no no 21.85s
✅ REPOSITORY git_diff yes no no 0.03s
✅ REPOSITORY grype yes no no 109.72s
⚠️ REPOSITORY osv-scanner yes 1 4 1.82s
✅ REPOSITORY secretlint yes no no 1.38s
✅ REPOSITORY syft yes no no 5.57s
✅ REPOSITORY trivy yes no no 14.08s
✅ REPOSITORY trivy-sbom yes no no 0.47s
✅ REPOSITORY trufflehog yes no no 5.43s
✅ SPELL lychee 125 0 0 10.64s
✅ YAML prettier 36 0 0 0 1.54s
✅ YAML v8r 36 0 0 12.2s
✅ YAML yamllint 36 0 0 1.35s

Detailed Issues

⚠️ REPOSITORY / osv-scanner - 1 error
warning: Package 'minimatch@10.2.6' is vulnerable to 'CVE-2026-118764' (also known as 'GHSA-g3hv-75wg-m249').
 = CVE-2026-118764: minimatch has potential denial of service via globparts amplification
 = minimatch 10.2.6 contains an attacker-controlled pattern amplification in firstPhasePreProcess() when optimizationLevel >= 2.
   
   researcher: Reid Downing (b1nv0id)
   files: 
   
   - src/index.ts: 690-770
   - src/index.ts: 780-796
   
   issue:
   input-controlled growth of globParts during first-phase preprocessing, followed by quadratic second-phase processing.
   
   the **/../ transformation clones a globparts entry and appends the clone to the same globparts array:

parts.splice(gs, 1)
const other = parts.slice(0)
other[gs] = '**'
globParts.push(other)

this occurs inside a do...while (didSomething) fixpoint loop, allowing generated entries to be processed again.

the expanded globparts array is subsequently processed by secondPhasePreProcess(), which performs pairwise comparisons:

for (let i = 0; i < globParts.length - 1; i++) {
for (let j = i + 1; j < globParts.length; j++) {
this.partsMatch(...)
}
}

an attacker who can supply a glob pattern can therefore cause additional pattern entries to be generated during preprocessing, followed by quadratic pairwise processing, resulting in excessive cpu and memory consumption.

warning: Package 'minimatch@10.2.4' is vulnerable to 'CVE-2026-118764' (also known as 'GHSA-g3hv-75wg-m249').
= CVE-2026-118764: minimatch has potential denial of service via globparts amplification
= minimatch 10.2.6 contains an attacker-controlled pattern amplification in firstPhasePreProcess() when optimizationLevel >= 2.

researcher: Reid Downing (b1nv0id)
files: 

- src/index.ts: 690-770
- src/index.ts: 780-796

issue:
input-controlled growth of globParts during first-phase preprocessing, followed by quadratic second-phase processing.

the **/../ transformation clones a globparts entry and appends the clone to the same globparts array:

parts.splice(gs, 1)
const other = parts.slice(0)
other[gs] = '**'
globParts.push(other)

this occurs inside a do...while (didSomething) fixpoint loop, allowing generated entries to be processed again.

the expanded globparts array is subsequently processed by secondPhasePreProcess(), which performs pairwise comparisons:

for (let i = 0; i < globParts.length - 1; i++) {
for (let j = i + 1; j < globParts.length; j++) {
this.partsMatch(...)
}
}

an attacker who can supply a glob pattern can therefore cause additional pattern entries to be generated during preprocessing, followed by quadratic pairwise processing, resulting in excessive cpu and memory consumption.

warning: Package 'bare-metal@0.2.5' is vulnerable to 'RUSTSEC-2026-0110'.
= RUSTSEC-2026-0110: bare-metal is deprecated
= The `bare-metal` crate has been deprecated and archived.

For `Mutex` and `CriticalSection`, see the [`critical-section`](https://crates.io/crates/critical-section) crate instead.

warning: Package 'bare-metal@0.2.5' is vulnerable to 'RUSTSEC-2026-0110'.
= RUSTSEC-2026-0110: bare-metal is deprecated
= The `bare-metal` crate has been deprecated and archived.

For `Mutex` and `CriticalSection`, see the [`critical-section`](https://crates.io/crates/critical-section) crate instead.

warning: 4 warnings emitted

See detailed reports in MegaLinter artifacts

You could have the same capabilities but better runtime performances if you use a MegaLinter flavor:

Your project could benefit from a custom flavor, which would allow you to run only the linters you need, and thus improve runtime performances. (Skip this info by defining FLAVOR_SUGGESTIONS: false)

  • Documentation: Custom Flavors
  • Command: npx mega-linter-runner@10.1.0 --custom-flavor-setup --custom-flavor-linters ACTION_ACTIONLINT,DOCKERFILE_HADOLINT,JSON_V8R,JSON_PRETTIER,JSON_NPM_PACKAGE_JSON_LINT,MARKDOWN_MARKDOWNLINT,MARKDOWN_MARKDOWN_TABLE_FORMATTER,REPOSITORY_CHECKOV,REPOSITORY_GIT_DIFF,REPOSITORY_BETTERLEAKS,REPOSITORY_GRYPE,REPOSITORY_OSV_SCANNER,REPOSITORY_SECRETLINT,REPOSITORY_SYFT,REPOSITORY_TRIVY,REPOSITORY_TRIVY_SBOM,REPOSITORY_TRUFFLEHOG,SPELL_LYCHEE,YAML_PRETTIER,YAML_YAMLLINT,YAML_V8R

MegaLinter is provided by OX Security
Show us your support by starring ⭐ the repository

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The host-compilation acceptance scenario is grouped under the ARM Cortex rule, undermining the traceability this PR aims to improve.

No open issues 🎉

What changed in this PR

This PR improves per-flavor requirements traceability by centralizing requirement definitions, adding Rust test links, and filtering acceptance scenarios by container flavor.

Changes:

  • Consolidates requirements under docs/requirements and updates related references.
  • Passes flavor tags through acceptance-test workflows and Playwright configuration.
  • Adds test source locations to generated documentation and distinguishes declared traceability from execution results.
File Description
test/​rust/​integration-tests.bats Adds requirement-trace annotations to existing tests.
test/​embedded-cpp/​features/​compilation.feature Removes the relocated ARM compilation requirement.
test/​cpp/​features/​playwright.config.ts Uses centralized features and environment-based tag filtering.
docs/​templates/​software-verification-plan.md.j2 Includes test source locations.
docs/​templates/​requirements-traceability-matrix.md.j2 Shows stable IDs, source locations, and declared traceability.
docs/​requirements/​static-dynamic-analysis.feature Centralizes analysis requirements and the formatting scenario.
docs/​requirements/​security.feature Centralizes supply-chain security requirements.
docs/​requirements/​maintainability.feature Centralizes maintainability requirements.
docs/​requirements/​embedded-debugging.feature Centralizes the firmware-upload requirement.
docs/​requirements/​debugging.feature Centralizes debugging requirements.
docs/​requirements/​compilation.feature Adds the relocated ARM Cortex compilation requirement.
docs/​requirements/​compatibility.feature Centralizes platform and tooling compatibility requirements.
docs/​adr/​0000-record-architecture-decisions.md Updates the maintainability requirement link.
.github/​workflows/​wc-document-generation.yml Removes the sample model before compilation.
.github/​workflows/​wc-build-push-test.yml Accepts and forwards the acceptance-test tag.
.github/​workflows/​wc-acceptance-test.yml Requires a flavor tag and exposes it to acceptance tests.
.github/​workflows/​build-push-test.yml Supplies the matrix flavor tag.

Review effort: Balanced

@github-actions

github-actions Bot commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

📦 Container Size Analysis

Note

Comparing ghcr.io/philips-software/amp-devcontainer-docs:edge ➔ ghcr.io/philips-software/amp-devcontainer-docs:pr-1509

📈 Size Comparison Table

OS/Platform Previous Current Change Trend
linux/amd64 227.22 MB 227.23 MB +7.48 kB (+0%) 🔼
linux/arm64 222.18 MB 222.18 MB 471 B (0%) 🔽

@github-actions

github-actions Bot commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

📦 Container Size Analysis

Note

Comparing ghcr.io/philips-software/amp-devcontainer-rust:edge ➔ ghcr.io/philips-software/amp-devcontainer-rust:pr-1509

📈 Size Comparison Table

OS/Platform Previous Current Change Trend
linux/amd64 451.93 MB 451.94 MB +6.61 kB (+0%) 🔼
linux/arm64 391.73 MB 391.73 MB 714 B (0%) 🔽

@github-actions

github-actions Bot commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

📦 Container Size Analysis

Note

Comparing ghcr.io/philips-software/amp-devcontainer-embedded-rust:edge ➔ ghcr.io/philips-software/amp-devcontainer-embedded-rust:pr-1509

📈 Size Comparison Table

OS/Platform Previous Current Change Trend
linux/amd64 518.67 MB 518.67 MB +5.76 kB (+0%) 🔼
linux/arm64 458.57 MB 458.57 MB 848 B (0%) 🔽

@github-actions

github-actions Bot commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

📦 Container Size Analysis

Note

Comparing ghcr.io/philips-software/amp-devcontainer-cpp:edge ➔ ghcr.io/philips-software/amp-devcontainer-cpp:pr-1509

📈 Size Comparison Table

OS/Platform Previous Current Change Trend
linux/amd64 423.7 MB 423.71 MB +7.1 kB (+0%) 🔼
linux/arm64 401.83 MB 401.83 MB +1.22 kB (+0%) 🔼

@github-actions

github-actions Bot commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

📦 Container Size Analysis

Note

Comparing ghcr.io/philips-software/amp-devcontainer-embedded-cpp:edge ➔ ghcr.io/philips-software/amp-devcontainer-embedded-cpp:pr-1509

📈 Size Comparison Table

OS/Platform Previous Current Change Trend
linux/amd64 625.86 MB 625.86 MB +4.89 kB (+0%) 🔼
linux/arm64 603.22 MB 603.22 MB +1.62 kB (+0%) 🔼

@github-actions

github-actions Bot commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

Test Results

 25 files  ±0   25 suites  ±0   12m 25s ⏱️ + 2m 57s
 57 tests ±0   57 ✅ ±0  0 💤 ±0  0 ❌ ±0 
297 runs  ±0  297 ✅ ±0  0 💤 ±0  0 ❌ ±0 

Results for commit 4b723d8. ± Comparison against base commit 82c881a.

♻️ This comment has been updated with latest results.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues are recorded for the requirements traceability and flavor-specific acceptance test changes.

No open issues 🎉

Review effort: Balanced

Ron (rjaegers) and others added 12 commits November 6, 2026 11:41
Show full source references in the SVP and RTM and distinguish declared traceability from execution evidence.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Keep full procedure descriptions in the SVP and link to unique requirement/procedure anchors from the RTM.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Use stable test IDs and a textual SVP reference instead of Markdown cross-document links and HTML anchors.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Keep full requirement descriptions in the SRS and preserve capability headings for context.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Compile the SBDL model from product feature/test sources and the two annotated workflows instead of recursively ingesting the repository, keeping the documentation fixture out of the supplier model.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Enumerate all test and workflow source files while excluding only the standalone sample SBDL fixture. Preserve the test model instead of narrowing directory roots.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Remove only the documentation fixture from the ephemeral checkout before invoking the SBDL compiler recursively. Keep compiler output outside the scanned tree until compilation succeeds.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Relocate requirement-only compatibility, maintainability, security, and debugging feature sources into docs/requirements while leaving executable scenarios under test.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Keep flavor-specific BDD scenarios and their tags under test while moving compilation and analysis declarations to docs/requirements.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Keep flavor-tagged Gherkin scenarios alongside their requirement declarations and filter acceptance test generation using the target flavor tag.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Ron (rjaegers) and others added 3 commits November 6, 2026 11:41
Update the BDD watcher to monitor docs/requirements and the actual Playwright harness paths.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Signed-off-by: Ron <45816308+rjaegers@users.noreply.github.com>
@rjaegers
Ron (rjaegers) force-pushed the docs/improve-per-flavor-traceability branch from 48a2b19 to 4b723d8 Compare November 6, 2026 10:41
@sonarqubecloud

sonarqubecloud Bot commented Nov 6, 2026

Copy link
Copy Markdown

@rjaegers
Ron (rjaegers) deployed to acceptance-testing November 6, 2026 10:50 — with GitHub Actions Active

This branch was successfully deployed

1 active deployment
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants